Spatio-Temporal Retinex-like Envelope with Total Variation

Many algorithms for spatial color correction of digital images have been proposed in the past. Some of the most recently developed algorithms use stochastic sampling of the image in order to obtain maximum and minimum envelope functions. The envelopes are in turn used to guide the color adjustment of the entire image. In this paper, we propose to use a variational method instead of the stochastic sampling to compute the envelopes. A numerical scheme for solving the variational equations is outlined, and we conclude that the variational approach is computationally more efficient than using stochastic sampling. A perceptual experiment with 20 observers and 13 images is carried out in order to evaluate the quality of the resulting images with the two approaches. There is no significant difference between the variational approach and the stochastic sampling when it comes to overall image quality as judged by the observers. However, the observed level of noise in the images is significantly reduced by the variational approach. Introduction A great amount of research has been done on Human Visual System (HVS), which is quite difficult to mimick as the HVS has complex and robust mechanisms to acquire useful informations from the physical environment. In particular the color of an area in a visual scene is heavily influenced by the chromatic content of the other areas of the scene. This psychophysiological phenomenon is known as locality of color perception. One of the earliest models able to deal with locality of perception is Retinex, proposed by Land and McCann in 1971 [14], which is an image processing method that exhibits some behaviors similar to the HVS. The scientific community has continued to be interested in this model and its various applications, as reported in [17, 16]. In the basic Land and McCann implementation of Retinex, locality is achieved by long paths scanning across images. Different implementations and analysis followed after this first work. These can be divided into two major groups, and they differ in the way they achieve locality. The first group explore the image using paths or computing ratios with neighbors in a multilevel framework [7, 13, 15, 21, 8, 4]. and recent approaches work using in particular Brownian motions models [6, 18]. The second group computes values over the image with convolution mask or weighting distances [11, 1, 10, 5, 20]. A recent implementation, constructed to investigate the effects of different spatial samplings, replaces paths with random sprays, i.e. two-dimensional point distributions across the image, hence the name ”Random Spray Retinex” (RSR) [19]. In a follow-up, Kolås et al. [12] developed the ”Spatio-Temporal Retinex-like Envelope with Stochastic Sampling” (STRESS) framework, where the random sprays are used to calculate two envelope functions representing the local reference black and white points. Both algorithms need a high density of samples in order to lower the amount of noise, but they never sample the whole image in order to keep a local effect. Furthermore the number of sampling points needed increases drastically when increasing the image size and consequently also the computational time. In this work, we propose and test an alternative method for calculating the two envelope functions of STRESS, replacing the the stochasting sampling with a constrained total variation method. We want to emphasize that although much of idea is the same, it is not just another implementation of STRESS as the two algorithms follow a different strategy for calculating the envelopes and they show different behaviors. In order to give to the reader a complete and detailed overview, STRESS will be described in the next section, followed by our new proposal. Afterwards, a description of the method of evaluation of our proposal in addition to some implementation details are presented. Finally, experimental results are shown and conclusions are drawn. The STRESS Algorithm The STRESS (Spatio-Temporal Retinex-like Envelope with Stochastic Sampling) algorithm developed by Kolås et al. [12] aims to reproduce some of the adjustment mechanisms typical for the Human Visual System. The central part of the STRESS algorithm is to calculate, for each pixel, the local reference black and white points in each chromatic channel. This is done through calculating two envelope functions, the maximum and minimum envelopes, containing the image signal. The two envelopes, denoted as Emax and Emin, are slowly varying functions, such that the image signal is always in between the envelopes or equal to one of them. In particular the two envelopes should have the following characteristics: 1) following the signal; 2) being smooth; 3) being edge preserving; 4) touching the global maximum of the image for Emax, while the global minimum for Emin. For each pixel p0, the two envelopes are estimated using a random spray modeled as follows: Emin = p0 −v r, (1a) Emax = p0 +(1− v r) = Emin + r (1b)